The aim of this paper by Songwei Ge,Vasu Singla, R. Basri and David Jacobs, is to show that shift invariance which is a useful property of CNNs can lead to greater sensitivity to adversarial attacks.
The first experiment is a simple example, a two-class classification problem. Each class consists of a single image: a white or black dot on a gray background. Two types of models are trained, a fully connected (FC) network and a convolutional neural network (CNN), with the CNN designed to be fully shift-invariant.
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from tqdm import tqdm
from functions.dots_models import *
%matplotlib inline
%load_ext autoreload
%autoreload 2
input_shape = (17, 17)
im_1 = np.zeros(input_shape)+0.5
im_2 = im_1.copy()
im_1[7][7]+=0.5
im_2[7][7]-=0.5
fig, axes = plt.subplots(1, 2)
cax1 = axes[0].imshow(im_1, cmap='gray', vmin=0, vmax=1)
axes[0].set_title("White Dot") #1
cax2 = axes[1].imshow(im_2, cmap='gray', vmin=0, vmax=1)
axes[1].set_title("Black Dot") #0
fig.colorbar(cax2, ax=axes)
plt.show()
# Creating dataset
data = np.stack([np.expand_dims(im_1, axis=-1), np.expand_dims(im_2, axis=-1)], axis=0)
y = np.array([1, 0])
Let's import the models from dot_models.py. The simple models are:
fcn_model: a Fully Connected Network (FCN) created by create_fcn, it starts with a BatchNormalization then a Flatten followed by a Dense layer of 512 units with a ReLU activation function and finally a Dense layer with 2 units and the softmax activation.
small_cnn_model_padding created by create_small_cnn. A simple Convolutional Neural Network (CNN) model that includes circular padding in the input. It begins with a BatchNormalization then a Lambda layer to apply circular padding. Then a Conv2D layer with 512 filters and a kernel size of 17 (k). A ReLU activation function, and lastly a Dense layer with 2 units and the softmax activation.
The previous two models are the ones used in the paper, the next one is added for comparison.
We can show the architectures of the models visually instead.
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
from PIL import ImageFont
Visual_Keras = True
# Set Visual_Keras = True to plot the keras graphs
if Visual_Keras:
import visualkeras
else:
from keras.utils.vis_utils import plot_model
k = 17
small_cnn_model = create_small_cnn_np(k)
if Visual_Keras:
visualkeras.layered_view(small_cnn_model, legend=True)
else:
plot_model(small_cnn_model, show_shapes=True, show_layer_names=True)
small_cnn_model_padding = create_small_cnn(k)
if Visual_Keras:
visualkeras.layered_view(small_cnn_model_padding, legend=True)
else:
plot_model(small_cnn_model_padding, show_shapes=True, show_layer_names=True)
fcn_model = create_fcn(k)
if Visual_Keras:
visualkeras.layered_view(fcn_model, legend=True)
else:
plot_model(fcn_model, show_shapes=True, show_layer_names=True)
from tensorflow.keras.losses import SparseCategoricalCrossentropy
from tensorflow.keras.optimizers import SGD
criterion = SparseCategoricalCrossentropy()
batch_X, batch_y = tf.constant(data, dtype=tf.float32), tf.constant(y, dtype=tf.int32)
small_cnn_model.summary()
LR = 0.01
opt = SGD(learning_rate=LR)
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
batch_normalization (BatchN (None, 17, 17, 1) 4
ormalization)
conv2d (Conv2D) (None, 1, 1, 512) 148480
activation (Activation) (None, 1, 1, 512) 0
global_average_pooling2d (G (None, 512) 0
lobalAveragePooling2D)
dense (Dense) (None, 2) 1026
=================================================================
Total params: 149,510
Trainable params: 149,508
Non-trainable params: 2
_________________________________________________________________
import importlib
from functions.dots_models import *
#importlib.reload(dots_models)
InteractiveShell.ast_node_interactivity = "last_expr"
train_dot_model(small_cnn_model, batch_X,batch_y,criterion,opt,LR)
Epoch: 100, Train Acc: 100.000, Loss: 0.006 Epoch: 200, Train Acc: 100.000, Loss: 0.003 Epoch: 300, Train Acc: 100.000, Loss: 0.002 Epoch: 400, Train Acc: 100.000, Loss: 0.001 Epoch: 500, Train Acc: 100.000, Loss: 0.001 Epoch: 600, Train Acc: 100.000, Loss: 0.001 Epoch: 700, Train Acc: 100.000, Loss: 0.001 Epoch: 800, Train Acc: 100.000, Loss: 0.001 Epoch: 900, Train Acc: 100.000, Loss: 0.001 Epoch: 1000, Train Acc: 100.000, Loss: 0.001 Epoch: 1100, Train Acc: 100.000, Loss: 0.001 Epoch: 1200, Train Acc: 100.000, Loss: 0.001 Epoch: 1300, Train Acc: 100.000, Loss: 0.001 Epoch: 1400, Train Acc: 100.000, Loss: 0.001 Epoch: 1500, Train Acc: 100.000, Loss: 0.000 Epoch: 1600, Train Acc: 100.000, Loss: 0.000 Epoch: 1700, Train Acc: 100.000, Loss: 0.000 Epoch: 1800, Train Acc: 100.000, Loss: 0.000 Epoch: 1900, Train Acc: 100.000, Loss: 0.000 Epoch: 2000, Train Acc: 100.000, Loss: 0.000
small_cnn_model_padding.summary()
LR = 0.01
opt = SGD(learning_rate=LR)
train_dot_model(small_cnn_model_padding, batch_X,batch_y,criterion,opt,LR)
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
batch_normalization_1 (Batc (None, 17, 17, 1) 4
hNormalization)
lambda (Lambda) (None, 33, 33, 1) 0
conv2d_1 (Conv2D) (None, 17, 17, 512) 148480
activation_1 (Activation) (None, 17, 17, 512) 0
global_average_pooling2d_1 (None, 512) 0
(GlobalAveragePooling2D)
dense_1 (Dense) (None, 2) 1026
=================================================================
Total params: 149,510
Trainable params: 149,508
Non-trainable params: 2
_________________________________________________________________
Epoch: 100, Train Acc: 100.000, Loss: 0.535
Epoch: 200, Train Acc: 100.000, Loss: 0.336
Epoch: 300, Train Acc: 100.000, Loss: 0.171
Epoch: 400, Train Acc: 100.000, Loss: 0.090
Epoch: 500, Train Acc: 100.000, Loss: 0.054
Epoch: 600, Train Acc: 100.000, Loss: 0.036
Epoch: 700, Train Acc: 100.000, Loss: 0.026
Epoch: 800, Train Acc: 100.000, Loss: 0.020
Epoch: 900, Train Acc: 100.000, Loss: 0.019
Epoch: 1000, Train Acc: 100.000, Loss: 0.019
Epoch: 1100, Train Acc: 100.000, Loss: 0.018
Epoch: 1200, Train Acc: 100.000, Loss: 0.018
Epoch: 1300, Train Acc: 100.000, Loss: 0.018
Epoch: 1400, Train Acc: 100.000, Loss: 0.017
Epoch: 1500, Train Acc: 100.000, Loss: 0.017
Epoch: 1600, Train Acc: 100.000, Loss: 0.016
Epoch: 1700, Train Acc: 100.000, Loss: 0.016
Epoch: 1800, Train Acc: 100.000, Loss: 0.016
Epoch: 1900, Train Acc: 100.000, Loss: 0.016
Epoch: 2000, Train Acc: 100.000, Loss: 0.016
fcn_model.summary()
LR = 0.01
opt = SGD(learning_rate=LR)
train_dot_model(fcn_model, batch_X,batch_y,criterion,opt,LR)
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
batch_normalization_2 (Batc (None, 17, 17, 1) 4
hNormalization)
flatten (Flatten) (None, 289) 0
dense_2 (Dense) (None, 512) 148480
dense_3 (Dense) (None, 2) 1026
=================================================================
Total params: 149,510
Trainable params: 149,508
Non-trainable params: 2
_________________________________________________________________
Epoch: 100, Train Acc: 100.000, Loss: 0.004
Epoch: 200, Train Acc: 100.000, Loss: 0.002
Epoch: 300, Train Acc: 100.000, Loss: 0.001
Epoch: 400, Train Acc: 100.000, Loss: 0.001
Epoch: 500, Train Acc: 100.000, Loss: 0.001
Epoch: 600, Train Acc: 100.000, Loss: 0.001
Epoch: 700, Train Acc: 100.000, Loss: 0.000
Epoch: 800, Train Acc: 100.000, Loss: 0.000
Epoch: 900, Train Acc: 100.000, Loss: 0.000
Epoch: 1000, Train Acc: 100.000, Loss: 0.000
Epoch: 1100, Train Acc: 100.000, Loss: 0.000
Epoch: 1200, Train Acc: 100.000, Loss: 0.000
Epoch: 1300, Train Acc: 100.000, Loss: 0.000
Epoch: 1400, Train Acc: 100.000, Loss: 0.000
Epoch: 1500, Train Acc: 100.000, Loss: 0.000
Epoch: 1600, Train Acc: 100.000, Loss: 0.000
Epoch: 1700, Train Acc: 100.000, Loss: 0.000
Epoch: 1800, Train Acc: 100.000, Loss: 0.000
Epoch: 1900, Train Acc: 100.000, Loss: 0.000
Epoch: 2000, Train Acc: 100.000, Loss: 0.000
We get the same result as the paper code. Similar values of the loss (which was observed in their notebook Two_Class_White_Black.ipynb). We also observe that the cnn without padding has a similar training loss to the fcn
Now let's generate adversarial data on the three models:
For each of the three models, we perform adversarial attacks using the PGD algorithm for different epsilon values. We generate adversarial examples, then calculate the distance between the original and adversarial examples (the distance is not used here), and evaluate the accuracy of the models on the adversarial examples.
from art.attacks.evasion import ProjectedGradientDescent
from art.estimators.classification import TensorFlowV2Classifier
# Define the loss object for each classifier
small_cnn_padding_loss_object = tf.keras.losses.SparseCategoricalCrossentropy()
fcn_loss_object = tf.keras.losses.SparseCategoricalCrossentropy()
small_cnn_loss_object = tf.keras.losses.SparseCategoricalCrossentropy()
distances_small_cnn_model = []
distances_small_cnn_model_padding = []
distances_fcn_model = []
accuracies_small_cnn_model = []
accuracies_small_cnn_model_padding = []
accuracies_fcn_model = []
small_cnn_model.compile(optimizer='adam', loss=small_cnn_loss_object, metrics=['accuracy'])
classifier = TensorFlowV2Classifier(
model=small_cnn_model,
loss_object=small_cnn_loss_object,
input_shape=(17, 17, 1),
nb_classes=2, #10
clip_values=(0, 1),
)
eps_values = [0.001, 0.002, 0.005, 0.01, 0.05, 0.1, 0.3, 0.5]
for eps in eps_values:
adversary = ProjectedGradientDescent(estimator=classifier, eps=eps, max_iter=100,verbose=False)
small_cnn_adv_x = adversary.generate(x=batch_X.numpy(), y=batch_y.numpy())
distance_small_cnn_model = np.linalg.norm(batch_X - small_cnn_adv_x)
distances_small_cnn_model.append(distance_small_cnn_model)
accuracy = small_cnn_model.evaluate(small_cnn_adv_x, batch_y,verbose=0)
accuracy_value = accuracy[1] * 100
accuracies_small_cnn_model.append(accuracy_value)
print("Epsilon: {}, Robust Accuracy: {}".format(eps, accuracy_value))
Epsilon: 0.001, Robust Accuracy: 100.0 Epsilon: 0.002, Robust Accuracy: 100.0 Epsilon: 0.005, Robust Accuracy: 100.0 Epsilon: 0.01, Robust Accuracy: 100.0 Epsilon: 0.05, Robust Accuracy: 100.0 Epsilon: 0.1, Robust Accuracy: 100.0 Epsilon: 0.3, Robust Accuracy: 0.0 Epsilon: 0.5, Robust Accuracy: 0.0
fcn_model.compile(optimizer='adam', loss=fcn_loss_object, metrics=['accuracy'])
classifier = TensorFlowV2Classifier(
model=fcn_model,
loss_object=fcn_loss_object,
input_shape=(17, 17, 1),
nb_classes=10,
clip_values=(0, 1),
)
eps_values = [0.001, 0.002, 0.005, 0.01, 0.05, 0.1, 0.3, 0.5]
for eps in eps_values:
adversary = ProjectedGradientDescent(estimator=classifier, eps=eps, max_iter=100,verbose=False)
fcn_model_adv_x = adversary.generate(x=batch_X.numpy(), y=batch_y.numpy())
distance_fcn_model = np.linalg.norm(batch_X - fcn_model_adv_x)
distances_fcn_model.append(distance_fcn_model)
accuracy = fcn_model.evaluate(fcn_model_adv_x, batch_y,verbose=0)
accuracy_value = accuracy[1] * 100
accuracies_fcn_model.append(accuracy_value)
print("Epsilon: {}, Robust Accuracy: {}".format(eps, accuracy_value))
Epsilon: 0.001, Robust Accuracy: 100.0 Epsilon: 0.002, Robust Accuracy: 100.0 Epsilon: 0.005, Robust Accuracy: 100.0 Epsilon: 0.01, Robust Accuracy: 100.0 Epsilon: 0.05, Robust Accuracy: 0.0 Epsilon: 0.1, Robust Accuracy: 0.0 Epsilon: 0.3, Robust Accuracy: 0.0 Epsilon: 0.5, Robust Accuracy: 0.0
small_cnn_model_padding.compile(optimizer='adam', loss=small_cnn_padding_loss_object, metrics=['accuracy'])
classifier = TensorFlowV2Classifier(
model=small_cnn_model_padding,
loss_object=small_cnn_padding_loss_object,
input_shape=(17, 17, 1),
nb_classes=10,
clip_values=(0, 1),
)
eps_values = [0.001, 0.002, 0.005, 0.01, 0.05, 0.1, 0.3, 0.5]
for eps in eps_values:
adversary = ProjectedGradientDescent(estimator=classifier, eps=eps, max_iter=100,verbose=False)
small_cnn_model_padding_adv_x = adversary.generate(x=batch_X.numpy(), y=batch_y.numpy())
distance_small_cnn_model_padding = np.linalg.norm(batch_X - small_cnn_model_padding_adv_x)
distances_small_cnn_model_padding.append(distance_small_cnn_model_padding)
accuracy = small_cnn_model_padding.evaluate(small_cnn_model_padding_adv_x, batch_y,verbose=0)
accuracy_value = accuracy[1] * 100
accuracies_small_cnn_model_padding.append(accuracy_value)
print("Epsilon: {}, Robust Accuracy: {}".format(eps, accuracy_value))
Epsilon: 0.001, Robust Accuracy: 100.0 Epsilon: 0.002, Robust Accuracy: 100.0 Epsilon: 0.005, Robust Accuracy: 0.0 Epsilon: 0.01, Robust Accuracy: 0.0 Epsilon: 0.05, Robust Accuracy: 0.0 Epsilon: 0.1, Robust Accuracy: 0.0 Epsilon: 0.3, Robust Accuracy: 0.0 Epsilon: 0.5, Robust Accuracy: 0.0
We can see that the fcn model is more robust compared to the cnn (it needs a bigger epsilon value to affect its robustness). And this is the same as observed by the results from the paper's experiment which are below: For the CNN:
| Epsilon | Robust Accuracy |
|---|---|
| 0.001 | 100 |
| 0.002 | 100 |
| 0.005 | 0 |
| 0.01 | 0 |
| 0.05 | 0 |
| 0.1 | 0 |
| 0.3 | 0 |
| 0.5 | 0 |
For the FCN:
| Epsilon | Robust Accuracy |
|---|---|
| 0.001 | 100 |
| 0.002 | 100 |
| 0.005 | 100 |
| 0.01 | 100 |
| 0.05 | 0 |
| 0.1 | 0 |
| 0.3 | 0 |
| 0.5 | 0 |
We also observe that the CNN without padding's robustness is similar to the FCN robustness. To visualize this, we plot:
# Plot the accuracies for each model
plt.plot(eps_values, accuracies_small_cnn_model, label="small_cnn_model")
plt.plot(eps_values, accuracies_small_cnn_model_padding, label="small_cnn_model_padding")
plt.plot(eps_values, accuracies_fcn_model, label="fcn_model")
plt.xlabel("Epsilon")
plt.ylabel("Accuracy (%)")
plt.title("Accuracy of Adversarial Examples by Epsilon")
plt.legend()
plt.show()
plt.plot(np.log10(eps_values), accuracies_small_cnn_model, marker='o', label="small_cnn_model")
plt.plot(np.log10(eps_values), accuracies_small_cnn_model_padding, marker='o', label="small_cnn_model_padding")
plt.plot(np.log10(eps_values), accuracies_fcn_model, marker='o', label="fcn_model")
plt.xlabel("Epsilon")
plt.ylabel("Accuracy (%)")
plt.title("Accuracy of Adversarial Examples by Epsilon Logarithmic")
plt.legend()
plt.show()
Here we verify the shift invariance of the models, this test is not in the paper's experiments explicitly but it's a claim that we want to verify, we simply shift the original images then evaluate the accuracy of the models to the shifts.
shift_range = 4
num_shifts = 9
shift_values = np.linspace(-shift_range, shift_range, num_shifts)
models = [small_cnn_model, small_cnn_model_padding, fcn_model]
model_names = ['small_cnn_model', 'small_cnn_model_padding', 'fcn_model']
colors = ['red', 'green', 'blue']
for model, model_name, color in zip(models, model_names, colors):
accuracies = []
for shift in shift_values:
shifted = np.roll(batch_X, int(shift), axis=1)
accuracy = model.evaluate(shifted, batch_y, verbose=0)[1]
accuracies.append(accuracy)
plt.plot(shift_values, accuracies, marker='o', color=color, label=model_name)
plt.xlabel('Shift Value')
plt.ylabel('Accuracy')
plt.title('Accuracy Comparison for Shifted Images')
plt.grid(True)
plt.legend()
plt.show()
We can see that the CNN is the only one who is shift invariant and that the CNN without padding behaves like the FCN (just like we saw with both the training and the PGD attack). So here the padding is exactly what gives the CNN its shift invariance and we observed that the shift invariant CNN is less robust to the pgd attack, we will confirm that with other types of attack.
And here are the adversarial images:
titles = ['Original images', 'CNN', 'CNN without padding', 'FCN']
arrays = [np.squeeze(batch_X), np.squeeze(small_cnn_model_padding_adv_x), np.squeeze(small_cnn_adv_x), np.squeeze(fcn_model_adv_x)]
fig, axs = plt.subplots(4, 2, figsize=(10, 10))
for i, array in enumerate(arrays):
for j in range(2):
axs[i, j].imshow(array[j], cmap='gray', vmin=0, vmax=1)
axs[i, j].set_title(titles[i])
axs[i, j].axis('off')
plt.tight_layout()
plt.show()
# this visual effect doesnt show in distances
# np.linalg.norm(np.squeeze(batch_X) - np.squeeze(small_cnn_adv_x))
Now, to generate the plot for the distances, the PGD is not used because it is a bounded-norm attack. And to see the effect of the attack on the distance between the original images and the attack images we need a bounded-norm attack. The paper's experiment used the DDN attack: Decoupling direction and norm. Since the DDN is not available in the tensorflow framework, we did the test using the same DNN used by the paper using pytorch, the models are recreated and retrained in pytorch. It's produced using: scripts/DNN_plot.py. And here is the result:
from IPython.display import Image
PATH = "plots/"
Image(filename = PATH + "black_white_dot.png", width=12, height=8)
Which is similar to the plot of the paper:
Image(filename = PATH + "teaser.png", width=12, height=8)
As we mentionned before, formally we can categorize the attacks into two categories; the first one is bounded-norm attacks, and the second one is minimum-norm attacks. DeepFool, C&W, FMN, FAB, DDN, and ALMA are minimumnorm attacks, and FGSM, PGD, and momentum extension of PGD are bounded-norm attacks. Revisiting DeepFool: generalization and improvement This part is also not in the paper but it's to understand more the difference between the the FCN and CNN when it comes to the perturbation distance ie how much perturbation is needed to get the adversarial image. So we will be using Deep Fool which is a minimum-norm attack, and we expect the behavior to be somehow similar to the results from the DNN. We are also adding the CNN without padding to the comparaison.
from art.attacks.evasion import DeepFool
from art.estimators.classification import TensorFlowV2Classifier
distances_dict = dict()
for model in ['FC', 'Conv_no_padding', 'Conv_padding']:
distances_adv = []
print(model)
for i in tqdm(range(4,15)):
k = i * 2 + 1
input_shape = (k, k)
batch = create_dataset_dots(k)
#batch_X, batch_y = tf.constant(np.expand_dims(batch[0], -1), dtype=tf.float32), tf.constant(batch[1], dtype=tf.int32)
batch_X, batch_y = np.expand_dims(batch[0], -1), batch[1]
n_hidden = 100
kernel_size = k
criterion = SparseCategoricalCrossentropy()
LR = 0.01
opt = SGD(learning_rate=LR)
net = create_fcn(k, n_hidden)
if model== 'FC':
net=create_fcn(k)
elif model=='Conv_no_padding':
net=create_small_cnn_np(k)
elif model=='Conv_padding':
net=create_small_cnn(k)
train_dot_model_no_verbose(net, batch_X, batch_y, criterion,opt,LR)
#batch_X = tf.Variable(batch_X)
classifier = TensorFlowV2Classifier(
model=net,
loss_object=criterion,
input_shape=(input_shape[0],input_shape[1],1),
nb_classes=2,
clip_values=(0, 1),
)
adv_crafter = DeepFool(classifier,batch_size=2, verbose=False)
x_adv = adv_crafter.generate(x=batch_X, y=batch_y)
#small_cnn_adv_x = adversary.generate(x=batch_X.numpy(), y=batch_y.numpy())
distance_ = np.linalg.norm(batch_X - x_adv)
distances_adv.append(distance_)
distances_dict[model] = distances_adv
FC
0%| | 0/11 [00:00<?, ?it/s]
WARNING:tensorflow:Calling GradientTape.gradient on a persistent tape inside its context is significantly less efficient than calling it outside the context (it causes the gradient ops to be recorded on the tape, leading to increased CPU and memory usage). Only call GradientTape.gradient inside the context if you actually want to trace the gradient in order to compute higher order derivatives.
100%|██████████| 11/11 [10:00<00:00, 54.57s/it]
Conv_no_padding
100%|██████████| 11/11 [07:47<00:00, 42.48s/it]
Conv_padding
100%|██████████| 11/11 [09:37<00:00, 52.47s/it]
CAND_COLORS = ['#a6cee3', '#1f78b4', '#b2df8a', '#33a02c', '#fb9a99', '#e31a1c', '#fdbf6f', '#ff7f00', '#cab2d6', '#6a3d9a', '#90ee90', '#9B870C', '#2f4554', '#61a0a8', '#d48265', '#c23531']
colors1 = CAND_COLORS[::2]
colors2 = CAND_COLORS[1::2]
with plt.style.context('bmh'):
fig = plt.figure(dpi=250, figsize=(10, 5.5))
plt.clf()
ks = [i * 2 + 1 for i in range(4,15)]
ax = plt.subplot(111)
plt.plot(ks, distances_dict['FC'], marker='o', linewidth=3, markersize=8, label='FC', color=colors2[0], alpha=0.8)
plt.plot(ks, distances_dict['Conv_no_padding'], marker='^', linewidth=3, markersize=8, label='Conv_no_padding', color=colors2[2], alpha=0.8)
plt.plot(ks, distances_dict['Conv_padding'], marker="D", linewidth=3, markersize=8, label='Conv_padding', color=colors2[2], alpha=0.8)
plt.plot(ks, [1.0/(i*2+1) for i in range(4,15)], linestyle='dotted', marker='*', linewidth=3, markersize=8, label='CNTKKKK', color=colors2[1], alpha=0.8)
plt.xlabel("Image size")
plt.ylabel("Distance between original images and adversarial images")
box = ax.get_position()
ax.set_position([box.x0, box.y0 + box.height * 0.1, box.width, box.height * 0.9])
plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.178), ncol=3)
plt.savefig('FC_CNN_CNNnp_distances.png')
CAND_COLORS = ['#a6cee3', '#1f78b4', '#b2df8a', '#33a02c', '#fb9a99', '#e31a1c', '#fdbf6f', '#ff7f00', '#cab2d6', '#6a3d9a', '#90ee90', '#9B870C', '#2f4554', '#61a0a8', '#d48265', '#c23531']
colors1 = CAND_COLORS[::2]
colors2 = CAND_COLORS[1::2]
with plt.style.context('bmh'):
fig = plt.figure(dpi=250, figsize=(10, 5.5))
plt.clf()
ks = [i * 2 + 1 for i in range(4,15)]
ax = plt.subplot(111)
plt.plot(ks, distances_dict['Conv_padding'], marker="D", linewidth=3, markersize=8, label='Conv_padding', color=colors2[2], alpha=0.8)
plt.plot(ks, [1.0/(i*2+1) for i in range(0,11)], linestyle='dotted', marker='*', linewidth=3, markersize=8, label='CNTKKKK', color=colors2[1], alpha=0.8)
plt.xlabel("Image size")
plt.ylabel("Distance between original images and adversarial images")
box = ax.get_position()
ax.set_position([box.x0, box.y0 + box.height * 0.1, box.width, box.height * 0.9])
plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.178), ncol=3)
plt.savefig('CNN_distances.png')
The behavior is not entirely the same. But can lead to the same conclusion, the FCN needs a much bigger perturbation (in this case increasing with the image size) while the CNN needs a much smaller perturbation, which is even decreasing, it doesnt follow the theoritical plot as for the DDN. And again,the FCN and the CNN without padding again have a similar behavior
Here we use MNIST (LeCun et al., 2010). To train the models, and produce the attack data. For script for this is in: /scripts/full_mnist_gpu_2.py Briefly here is what the script does:
attack.generate(x=x_train) while for the PGD we used attack.generate(x=x_train[:10000]) and attack.generate(x=x_test[:5000]).We used LRZ AI Systems to run the scripts. More details about this will be in the repository.
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, ReLU, GlobalAveragePooling2D, Dense, Lambda, GlobalMaxPooling2D, Flatten
from art.estimators.classification import TensorFlowV2Classifier
from art.attacks.evasion import FastGradientMethod, ProjectedGradientDescentTensorFlowV2
import os
from tensorflow.keras import layers
import pickle
from art.utils import load_mnist
(x_train, y_train), (x_test, y_test), min_pixel_value, max_pixel_value = load_mnist()
class_names = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
num_images = 5
random_indexes = np.random.choice(len(x_test), num_images, replace=False)
images = x_test[random_indexes]
labels = y_test[random_indexes]
plt.figure(figsize=(10, 5))
for i in range(num_images):
plt.subplot(1, num_images, i+1)
plt.imshow(images[i].reshape(28, 28), cmap='gray')
plt.title(class_names[np.argmax(labels[i])])
plt.axis('off')
plt.show()
Here is a brief description of the models:
The next three models are added:
The output layer for all models consists of 10 units, corresponding to the 10 classes in the MNIST dataset.
The models that are used in the paper are Simple_FC and Simple_Conv, with different hyperparameters [simple_FC_256, simple_Conv_10_512, simple_FC_1024,simple_Conv_12_2048]. While the other models are added for comparison (to get more understanding), note that the max pooling was used by the paper's authors during tests (in their repo) but it is not used in the paper.
def circular_padding(x, padding_size):
# Perform circular padding on the input tensor
return tf.pad(x, [[0, 0], [padding_size, padding_size], [padding_size, padding_size], [0, 0]], mode='SYMMETRIC')
def simple_Conv(n_hidden, kernel_size=10, padding_size=-1):
if padding_size == -1:
padding_size = kernel_size // 2
model = Sequential()
model.add(Lambda(lambda x: circular_padding(x, padding_size), input_shape=(28, 28, 1)))
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='same'))
model.add(ReLU())
model.add(GlobalAveragePooling2D())
model.add(Dense(10))
return model
def simple_FC(n_hidden):
model = Sequential()
model.add(Flatten(input_shape=(28, 28)))
model.add(Dense(n_hidden, activation="relu"))
model.add(Dense(10))
return model
def simple_Conv_NL(n_hidden,kernel_size=10):
""" no lambda """
model = Sequential()
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='valid', input_shape=(28, 28, 1), activation='relu'))
model.add(GlobalAveragePooling2D())
model.add(Dense(10))
return model
def simple_Conv_max(n_hidden, kernel_size=10,padding_size=0):
model = Sequential()
model.add(Lambda(lambda x: circular_padding(x, padding_size), input_shape=(28, 28, 1)))
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='same', activation='relu'))
model.add(GlobalMaxPooling2D())
model.add(Dense(10))
return model
def simple_Conv_2():
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(10))
return model
LRZ = False
if LRZ:
base_path = "/dss/dsshome1/lxc0C/apdl010/Analysis-of-new-phenomena-in-deep-learning/Reproduction"
else:
base_path = "models/mnist_main"
# Load models
simple_FC_256 = tf.keras.models.load_model(base_path + '/mnist_model_0.h5', compile=False)
simple_Conv_10_512 = tf.keras.models.load_model(base_path+'/mnist_model_1.h5', compile=False )
simple_Conv_NL = tf.keras.models.load_model(base_path+'/mnist_model_2.h5', compile=False )
simple_Conv_max = tf.keras.models.load_model(base_path+'/mnist_model_3.h5', compile=False )
simple_FC_1024 = tf.keras.models.load_model(base_path+'/mnist_model_4.h5', compile=False )
simple_Conv_12_2048 = tf.keras.models.load_model(base_path+'/mnist_model_5.h5', compile=False )
simple_Conv_2 = tf.keras.models.load_model(base_path+'/mnist_model_6.h5', compile=False ) # there was a typo here mnist_model_5
model_names = {'simple_FC_256':simple_FC, 'simple_Conv_10_512':simple_Conv, 'simple_Conv_NL':simple_Conv_NL, 'simple_Conv_max':simple_Conv_max,
'simple_FC_1024':simple_FC, 'simple_Conv_12_2048':simple_Conv,'simple_Conv_2':simple_Conv_2}
models = [simple_FC_256, simple_Conv_10_512, simple_Conv_NL, simple_Conv_max, simple_FC_1024,simple_Conv_12_2048,simple_Conv_2]
models = [[models[i]] for i in range(len(models))]
All models were trained for 20 epochs using the ADAM optimizer, with a batch size of 200 and learning rate of 0.01. The learning rate is decreased by a factor of 10 at the 10th and 15th epoch. To evaluate the robustness of these models. We use PGD l2 and l∞ attacks with different epsilon values, a single random restart, 10 iterations and step-size of epsilon/5. Just like it is stated by the paper.
base_path = "data/adversarial_data_2"
# Load the attack data
attack_data = {}
attack_params = [[np.inf, [0.05, 0.1, 0.15, 0.2, 0.25, 0.3]],[2, [0.5, 1, 1.5, 2.5, 3]]]
attack_name = "ProjectedGradientDescentTensorFlowV2"
for model_name, model in zip(model_names,models):
for norm, epsilons in attack_params:
for epsilon in epsilons:
for data_type in ["train", "test"]:
file_name = base_path + f"/2_{model_name}_{attack_name}_{epsilon}_{data_type}.npz"
try:
attack_data[(model_name, norm, epsilon,data_type)] = (model,np.load(file_name))
except FileNotFoundError:
print(file_name + " data not found ")
We evaluate the accuracy on test data, to do it for training data use: data_type == "train", the results should be similar.
norms = [attack_params[0][0], attack_params[1][0]]
epsilons = {
attack_params[0][0]: attack_params[0][1],
attack_params[1][0]: attack_params[1][1]
}
# Getting accuracies on normal data (epsilon = 0 as a notation)
accuracy_data = {norm: {model: [] for model in list(model_names.keys())} for norm in norms}
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(x_test,verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
accuracy_data[norm][model_name].append((0, accuracy))
else:
continue
# Getting accuracies on adversarial data (epsilon > 0)
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(data[f'x_{data_type}_attack'],verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(data[f'y_{data_type}_attack'], axis=1)) / len(data[f'y_{data_type}_attack'])
accuracy_data[norm][model_name].append((epsilon, accuracy))
else:
continue
100%|██████████| 154/154 [04:20<00:00, 1.69s/it] 100%|██████████| 154/154 [02:48<00:00, 1.09s/it]
for norm in norms:
plt.figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
plt.plot(eps, accs, marker='o', label=model_name)
plt.xlabel('Epsilon')
plt.ylabel('Accuracy')
plt.title(f"Accuracy on Attack Data (Norm={norm})")
plt.legend()
plt.show()
import plotly.graph_objects as go
for norm in norms:
fig = go.Figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
# Set the line style based on the model name
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=eps, y=accs, mode='lines+markers', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis_title='Epsilon',
yaxis_title='Accuracy',
title=f'Accuracy on Attack Data (Norm={norm})',
showlegend=True
)
fig.show()
For the reproduction part we get exactly as expected, if we select the models [simple_FC_256, simple_Conv_10_512, simple_FC_1024,simple_Conv_12_2048] in the interactive html plot, the plot we get is similar to the one produced by the paper. We observe that although the clean accuracy for the models is similar on the dataset, FC networks are more robust than Shift Invariant CNNs, especially for large epsilon values, for both norms.
For the additional models, it seems that max pooling makes the models more robust but it is still less robust than the FCN. And the CNN with no padding is a little more robust for smaller epsilon values, while the CNN with multiple layers has an unexpected behavior, it seems more robust maybe 10 iterations are not enough. We will see that it has a similar behavior for the other dataset, and we need to explore this in the extention part
Let's evaluate shift invariance using the shifting method.
def shift_images(images, shift, axis=0):
shifted_images = np.roll(images, shift, axis=axis)
return shifted_images
def evaluate_shift_invariance(model, x_test, y_test, shifts, axis=0):
accuracies = []
for shift in tqdm(shifts):
shifted_images = shift_images(x_test, shift, axis=axis)
predictions = model.predict(shifted_images,verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
accuracies.append(accuracy)
return accuracies
shifts = range(10, -11, -1)
results = []
for model, model_name in zip(models, model_names):
model = model[0]
accuracy_axis1 = evaluate_shift_invariance(model, x_test, y_test, shifts, axis=1)
results.append((model_name, accuracy_axis1))
for model_name, accuracy_axis1 in results:
plt.plot(shifts, accuracy_axis1, label=model_name)
plt.xlabel('Shift')
plt.ylabel('Accuracy')
plt.legend()
plt.title('Shift Invariance Evaluation')
plt.tight_layout()
plt.show()
100%|██████████| 21/21 [00:38<00:00, 1.84s/it] 100%|██████████| 21/21 [01:03<00:00, 3.00s/it] 100%|██████████| 21/21 [00:26<00:00, 1.25s/it] 100%|██████████| 21/21 [01:06<00:00, 3.16s/it] 100%|██████████| 21/21 [00:21<00:00, 1.03s/it] 100%|██████████| 21/21 [06:01<00:00, 17.19s/it] 100%|██████████| 21/21 [06:03<00:00, 17.33s/it]
fig = go.Figure()
for model_name, accuracy_axis1 in results:
shifts_abs = []
results_abs_k = []
for k in range(len(shifts) // 2 + 1):
shifts_abs.append(shifts[k])
results_abs_k.append((accuracy_axis1[k] + accuracy_axis1[-k - 1]) / 2)
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=shifts_abs, y=results_abs_k, mode='lines', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis=dict(title='Shift'),
yaxis=dict(title='Accuracy'),
title='Shift Invariance Evaluation',
showlegend=True
)
fig.show()
We can see that the FC models are not shift invariant compared to the CNNS, but for the CNNs the shift invariance isn't necessarily related to the robustness (using this shift method).
In the paper, shift invariance is evaluated using shift consistency scores. The consistency score calculates the percentage of the time that the model preserves its predicted labels when a random shift is applied to the image Making Convolutional Networks Shift-Invariant Again . The algorithm to compute it is evaluate_shift_consistency and can be found in utils.py . Note that shift consistency does not measure the accuracy of the model if the data is shifted (which is what the shifting method computes) but it evaluates the consistency of the model to shifting.
from tabulate import tabulate
from functions.utils import evaluate_shift_consistency
# getting consistency scores
model_consistency_dict = evaluate_shift_consistency(models,list(model_names.keys()), x_test)
# Print the model consistency dictionary as a table
table = []
for model_name, model_consistency in model_consistency_dict.items():
table.append([model_name, "{:.2f}%".format(100. * model_consistency)])
headers = ["Model Name", "Consistency"]
print(tabulate(table, headers, tablefmt="grid"))
+---------------------+---------------+ | Model Name | Consistency | +=====================+===============+ | simple_FC_256 | 28.64% | +---------------------+---------------+ | simple_Conv_10_512 | 46.51% | +---------------------+---------------+ | simple_Conv_NL | 55.27% | +---------------------+---------------+ | simple_Conv_max | 63.13% | +---------------------+---------------+ | simple_FC_1024 | 28.01% | +---------------------+---------------+ | simple_Conv_12_2048 | 50.19% | +---------------------+---------------+ | simple_Conv_2 | 49.70% | +---------------------+---------------+
We note that these scores are much smaller for the FCNs (28%) than for the CNNs(46-63%). If the consistency score should be inversly related to the invariance, then the order of robustness should be: simple_FC_1024, simple_FC_256, simple_Conv_10_512,simple_Conv_2, simple_Conv_12_2048, simple_Conv_NL,simple_Conv_max While the actual order of robustness for example for norm=∞ and epsilon=2: simple_Conv_2, simple_FC_1024, simple_FC_256, simple_Conv_max, simple_Conv_10_512, simple_Conv_12_2048, simple_Conv_NL
The models used in the paper, respect this order.
This section is not a reproduction of the papers results, but it was discussed during the presentations of progress, so we included it in the notebook.
norm_ = np.inf
epsilon_ = 0.1
data_type_ = "test"
accuracy_data_0_1_pgd = {model_name: {model: [] for model in model_names.keys()} for model_name in model_names.keys()}
adversarial_data_x = {}
adversarial_data_y = {}
for original_model in model_names:
for (model_name, norm, epsilon, data_type), (model, data) in attack_data.items():
if (model_name, norm, epsilon, data_type) == (original_model, norm_, epsilon_, data_type_):
adversarial_data_x[original_model] = data[f'x_{data_type}_attack']
adversarial_data_y[original_model] = data[f'y_{data_type}_attack']
else:
continue
for original_model in tqdm(model_names):
for (model_name, norm, epsilon, data_type), (model, data) in attack_data.items():
if data_type == data_type_ and epsilon == epsilon_ and norm == norm_:
model = model[0]
predictions = model.predict(adversarial_data_x[original_model], verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(adversarial_data_y[original_model], axis=1)) / len(adversarial_data_y[original_model])
accuracy_data_0_1_pgd[original_model][model_name].append((norm,epsilon,accuracy))
else:
continue
100%|██████████| 7/7 [02:26<00:00, 20.98s/it]
fig, axes = plt.subplots(3, 3, figsize=(15,15))
axes = axes.flatten()
for i, (model_name, model_data) in enumerate(accuracy_data_0_1_pgd.items()):
models = list(model_data.keys())
accuracies = [item[0][2] for item in model_data.values()]
ax = axes[i]
ax.bar(models, accuracies)
ax.set_xlabel('Models')
ax.set_ylabel('Accuracy')
ax.set_title(f'Accuracy on data generated by: {model_name}')
ax.tick_params(axis='x', rotation=45)
plt.tight_layout()
plt.show()
Instead of a fixed epsilon, let's plot for all epsilons.
norm_ = np.inf
data_type_ = "test"
epsilons = {
attack_params[0][0]: attack_params[0][1],
attack_params[1][0]: attack_params[1][1]
}
accuracy_data_pgd = {model_name: {model: [] for model in model_names.keys()} for model_name in model_names.keys()}
adversarial_data_x = {}
adversarial_data_y = {}
for original_model in model_names:
for (model_name, norm, epsilon, data_type), (model, data) in attack_data.items():
if (model_name, norm, data_type) == (original_model, norm_, data_type_):
adversarial_data_x[original_model,epsilon] = data[f'x_{data_type}_attack']
adversarial_data_y[original_model,epsilon] = data[f'y_{data_type}_attack']
else:
continue
for original_model in tqdm(model_names):
for (model_name, norm, epsilon, data_type), (model, data) in attack_data.items():
if data_type == data_type_ and norm == norm_:
model = model[0]
predictions = model.predict(adversarial_data_x[original_model,epsilon], verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(adversarial_data_y[original_model,epsilon], axis=1)) / len(adversarial_data_y[original_model,epsilon])
accuracy_data_pgd[original_model][model_name].append((norm, epsilon, accuracy))
else:
continue
100%|██████████| 7/7 [18:33<00:00, 159.05s/it]
# Plotting the accuracies for different epsilons
fig, axes = plt.subplots(len(model_names), 1, figsize=(8, 6*len(model_names)))
for i, (original_model, model_data) in enumerate(accuracy_data_pgd.items()):
ax = axes[i]
for model_name, accuracies in model_data.items():
epsilons = [entry[1] for entry in accuracies]
accuracy_values = [entry[2] for entry in accuracies]
ax.plot(epsilons, accuracy_values, marker='o', label=model_name)
ax.set_xlabel('Epsilon')
ax.set_ylabel('Accuracy')
ax.set_title(f'Accuracy vs Epsilon for {original_model}')
ax.legend()
plt.tight_layout()
plt.show()
We will go back to this in the extension part. Overall we can say that the FC models are robust against the CNN attack data (which is to be expected). And the more similar the architecture of the models the more transferable is the attack data, with the max pooling model being an outlier for this.
This part is also not a reproduction, it's more of a verification produced using full_mnist_gpu.py. Where we used the FGM attack instead of the PGD norm = 2 (for the FGM we produced the attack on the full data, as it's much faster to produce). We will also see the models, their accuracies and losses for the training. For the pgd we didn't limit the iterations to 10 but to 100 and we left the other parameters to default.
model_names = {'simple_FC_256':simple_FC, 'simple_Conv_10_512':simple_Conv, 'simple_Conv_NL':simple_Conv_NL, 'simple_Conv_max':simple_Conv_max,
'simple_FC_1024':simple_FC, 'simple_Conv_12_2048':simple_Conv}
LRZ = False
if LRZ:
base_path = "/dss/dsshome1/lxc0C/apdl010/Analysis-of-new-phenomena-in-deep-learning/Reproduction"
else:
base_path = "models/mnist_2"
simple_FC_256 = tf.keras.models.load_model(base_path + '/mnist_model_0.h5', compile=False)
simple_Conv_10_512 = tf.keras.models.load_model(base_path+'/mnist_model_1.h5', compile=False )
simple_Conv_NL = tf.keras.models.load_model(base_path+'/mnist_model_2.h5', compile=False )
simple_Conv_max = tf.keras.models.load_model(base_path+'/mnist_model_3.h5', compile=False )
simple_FC_1024 = tf.keras.models.load_model(base_path+'/mnist_model_4.h5', compile=False )
simple_Conv_12_2048 = tf.keras.models.load_model(base_path+'/mnist_model_5.h5', compile=False )
with open(base_path + '/mnist_acc_array.pkl', 'rb') as file:
acc_array = pickle.load(file)
with open(base_path + '/mnist_loss_array.pkl', 'rb') as file:
loss_array = pickle.load(file)
models = [simple_FC_256, simple_Conv_10_512, simple_Conv_NL, simple_Conv_max, simple_FC_1024,simple_Conv_12_2048]
models = [[models[i]] + [acc_array[i][0]] + [acc_array[i][1]] + [loss_array[i][0]] + [loss_array[i][1]] for i in range(len(models))]
Now, let's check the models and the losses and accuracies
InteractiveShell.ast_node_interactivity = "all"
visualkeras.layered_view(simple_FC_256, legend=True)
visualkeras.layered_view(simple_FC_1024, legend=True)
visualkeras.layered_view(simple_Conv_10_512, legend=True)
visualkeras.layered_view(simple_Conv_12_2048, legend=True)
InteractiveShell.ast_node_interactivity = "last_expr"
import plotly.graph_objects as go
import plotly.io as pio
fig_traces = []
for model_name, (_, test_acc, train_acc,_,_) in zip(model_names.keys(), models):
trace_train = go.Scatter(x=np.arange(len(train_acc)), y=train_acc, mode='lines', name=f'{model_name} (Train)')
trace_test = go.Scatter(x=np.arange(len(test_acc)), y=test_acc, mode='lines', name=f'{model_name} (Test)')
fig_traces.extend([trace_train, trace_test])
layout = go.Layout(
xaxis=dict(title='Epoch'),
yaxis=dict(title='Accuracy'),
title='Training and Validation Accuracy',
showlegend=True
)
fig = go.Figure(data=fig_traces, layout=layout)
pio.show(fig)
fig_traces = []
for model_name, (_,_,_,test_loss,train_loss) in zip(model_names.keys(), models):
trace_train = go.Scatter(x=np.arange(len(train_loss)), y=train_loss, mode='lines', name=f'{model_name} (Train)')
trace_test = go.Scatter(x=np.arange(len(test_loss)), y=test_loss, mode='lines', name=f'{model_name} (Test)')
fig_traces.extend([trace_train, trace_test])
layout = go.Layout(
xaxis=dict(title='Epoch'),
yaxis=dict(title='Loss'),
title='Training and Validation Loss',
showlegend=True
)
fig = go.Figure(data=fig_traces, layout=layout)
pio.show(fig)
base_path = "data/mnist_adversarial_data"
# Load the attack data
attack_data = {}
attack_params = [[np.inf, [0.05, 0.1, 0.15, 0.2, 0.25, 0.3]],[2, [0.5, 1, 1.5, 2.5, 3]]]
for model_name, model in zip(model_names,models):
for norm, epsilons in attack_params:
for epsilon in epsilons:
for data_type in ["train", "test"]:
if norm == np.inf:
attack_name = "ProjectedGradientDescentTensorFlowV2"
else:
attack_name = "FastGradientMethod"
file_name = base_path + f"/{model_name}_{attack_name}_{epsilon}_{data_type}.npz"
try:
attack_data[(model_name, norm, epsilon,data_type)] = (model,np.load(file_name))
except FileNotFoundError:
print(file_name + " data not found ")
norms = [attack_params[0][0], attack_params[1][0]]
epsilons = {
attack_params[0][0]: attack_params[0][1],
attack_params[1][0]: attack_params[1][1]
}
# Getting accuracies on normal data (epsilon = 0 as a notation)
accuracy_data = {norm: {model: [] for model in list(model_names.keys())} for norm in norms}
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(x_test,verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
accuracy_data[norm][model_name].append((0, accuracy))
else:
continue
# Getting accuracies on adversarial data (epsilon > 0)
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(data[f'x_{data_type}_attack'],verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(data[f'y_{data_type}_attack'], axis=1)) / len(data[f'y_{data_type}_attack'])
accuracy_data[norm][model_name].append((epsilon, accuracy))
else:
continue
100%|██████████| 132/132 [17:36<00:00, 8.00s/it] 100%|██████████| 132/132 [12:30<00:00, 5.69s/it]
for norm in norms:
plt.figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
plt.plot(eps, accs, marker='o', label=model_name)
plt.xlabel('Epsilon')
plt.ylabel('Accuracy')
plt.title(f"Accuracy on Attack Data (Norm={norm})")
plt.legend()
plt.show()
import plotly.graph_objects as go
for norm in norms:
fig = go.Figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
# Set the line style based on the model name
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=eps, y=accs, mode='lines+markers', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis_title='Epsilon',
yaxis_title='Accuracy',
title=f'Accuracy on Attack Data (Norm={norm})',
showlegend=True
)
fig.show()
The same observations hold for these different parameters and also for the FastGradientMethod attack (noted here as norm=2).
Here we will use the Fashion-MNIST dataset. We created the models and attack data using scripts/full_fashion_mnist_gpu_2.py. Using LRZ with DGX-1 P100 Architecture (4 to 16 gpus). For the created attack data we used x_train[:10000] and x_test[:5000].
Briefly here is what we do:
For the attack data creation:
fashion_mnist = tf.keras.datasets.fashion_mnist
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
x_train = x_train / 255.0
x_test = x_test / 255.0
x_train = x_train.reshape(-1, 28, 28, 1)
x_test = x_test.reshape(-1, 28, 28, 1)
# Convert the labels to one-hot encoding
y_train = tf.keras.utils.to_categorical(y_train, num_classes=10)
y_test = tf.keras.utils.to_categorical(y_test, num_classes=10)
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal',
'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
num_images = 5
random_indexes = np.random.choice(len(x_test), num_images, replace=False)
images = x_test[random_indexes]
labels = y_test[random_indexes]
plt.figure(figsize=(10, 5))
for i in range(num_images):
plt.subplot(1, num_images, i+1)
plt.imshow(images[i].reshape(28, 28), cmap='gray')
plt.title(class_names[np.argmax(labels[i])])
plt.axis('off')
plt.show()
The used models below are similar to the ones used with mnist. Here is a brief introduction to the models:
The output layer for all these models consists of 10 units, corresponding to the 10 classes in the MNIST dataset. And here we will introduce some new architectures that were not used in the mnist experiment:
The models used in the paper are: simple_FC_2_256, simple_Conv_10_512, simple_FC_3_512':simple_FC and simple_Conv_12_2048.
from tensorflow.keras.layers import LSTM, MaxPooling2D,Reshape
def circular_padding(x, padding_size):
# Perform circular padding on the input tensor
return tf.pad(x, [[0, 0], [padding_size, padding_size], [padding_size, padding_size], [0, 0]], mode='SYMMETRIC')
def simple_Conv(n_hidden, kernel_size=10, padding_size=-1):
if padding_size == -1:
padding_size = kernel_size // 2
model = Sequential()
model.add(Lambda(lambda x: circular_padding(x, padding_size), input_shape=(28, 28, 1)))
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='same'))
model.add(ReLU())
model.add(GlobalAveragePooling2D())
model.add(Dense(10))
return model
def simple_FC(n_hidden, n_units = 2):
model = Sequential()
model.add(Flatten(input_shape=(28, 28)))
for i in range(n_units):
model.add(Dense(n_hidden, activation="relu"))
model.add(Dense(10))
return model
def simple_Conv_NL(n_hidden,kernel_size=10):
""" no lambda """
model = Sequential()
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='valid', input_shape=(28, 28, 1), activation='relu'))
model.add(GlobalAveragePooling2D())
model.add(Dense(10))
return model
def simple_Conv_max(n_hidden, kernel_size=10,padding_size=0):
model = Sequential()
model.add(Lambda(lambda x: circular_padding(x, padding_size), input_shape=(28, 28, 1)))
model.add(Conv2D(n_hidden, kernel_size=kernel_size, padding='same', activation='relu'))
model.add(GlobalMaxPooling2D())
model.add(Dense(10))
return model
def simple_Conv_2():
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(10))
return model
def simple_FC_2():
model = Sequential()
model.add(Flatten(input_shape=(28, 28)))
model.add(Dense(256, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(10))
return model
def simple__RNN():
model = Sequential()
model.add(Reshape((28, 28), input_shape=(28, 28, 1)))
model.add(LSTM(128, return_sequences=True))
model.add(LSTM(128))
model.add(Dense(10))
return model
LRZ = False
if LRZ:
base_path = "/dss/dsshome1/lxc0C/apdl010/Analysis-of-new-phenomena-in-deep-learning/Reproduction"
else:
base_path = "models/fashion_mnist_main"
simple_FC_2_256 = tf.keras.models.load_model(base_path + '/fashion_mnist_simple_FC_2_256.h5', compile=False)
simple_Conv_10_512 = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_Conv_10_512.h5', compile=False )
simple_Conv_NL = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_Conv_NL.h5', compile=False )
simple_Conv_max = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_Conv_max.h5', compile=False )
simple_FC_3_512 = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_FC_3_512.h5', compile=False )
simple_Conv_12_2048 = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_Conv_12_2048.h5', compile=False )
simple__RNN = tf.keras.models.load_model(base_path+'/fashion_mnist_simple__RNN.h5', compile=False )
simple_FC_2 = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_FC_2.h5', compile=False )
simple_Conv_2 = tf.keras.models.load_model(base_path+'/fashion_mnist_simple_Conv_2.h5', compile=False )
model_names = {'simple_FC_2_256':simple_FC, 'simple_Conv_10_512':simple_Conv, 'simple_Conv_NL':simple_Conv_NL,
'simple_Conv_max':simple_Conv_max, 'simple_FC_3_512':simple_FC, 'simple_Conv_12_2048':simple_Conv,
'simple__RNN':simple__RNN , 'simple_FC_2':simple_FC_2, 'simple_Conv_2':simple_Conv_2}
models = [simple_FC_2_256, simple_Conv_10_512, simple_Conv_NL, simple_Conv_max, simple_FC_3_512,simple_Conv_12_2048, simple__RNN,simple_FC_2,simple_Conv_2]
models = [[models[i]] for i in range(len(models))]
base_path = "data/fashion_adversarial_data_2"
# Load the attack data
attack_data = {}
attack_params = [[np.inf, [0.05, 0.1, 0.15, 0.2, 0.25, 0.3]],[2, [0.5, 1, 1.5, 2.5, 3]]]
attack_name = "ProjectedGradientDescentTensorFlowV2"
for model_name, model in zip(model_names,models):
for norm, epsilons in attack_params:
for epsilon in epsilons:
for data_type in ["train", "test"]:
file_name = base_path + f"/2_{model_name}_{attack_name}_{epsilon}_{data_type}.npz"
try:
attack_data[(model_name, norm, epsilon,data_type)] = (model,np.load(file_name))
except FileNotFoundError:
print(file_name + " data not found ")
norms = [attack_params[0][0], attack_params[1][0]]
epsilons = {
attack_params[0][0]: attack_params[0][1],
attack_params[1][0]: attack_params[1][1]
}
# Getting accuracies on normal data (epsilon = 0 as a notation)
accuracy_data = {norm: {model: [] for model in list(model_names.keys())} for norm in norms}
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(x_test,verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
accuracy_data[norm][model_name].append((0, accuracy))
else:
continue
# Getting accuracies on adversarial data (epsilon > 0)
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(data[f'x_{data_type}_attack'],verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(data[f'y_{data_type}_attack'], axis=1)) / len(data[f'y_{data_type}_attack'])
accuracy_data[norm][model_name].append((epsilon, accuracy))
else:
continue
100%|██████████| 198/198 [37:12<00:00, 11.27s/it] 100%|██████████| 198/198 [13:41<00:00, 4.15s/it]
for norm in norms:
plt.figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
plt.plot(eps, accs, marker='o', label=model_name)
plt.xlabel('Epsilon')
plt.ylabel('Accuracy')
plt.title(f"Accuracy on Attack Data (Norm={norm})")
plt.legend()
plt.show()
for norm in norms:
fig = go.Figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
# Set the line style based on the model name
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=eps, y=accs, mode='lines+markers', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis_title='Epsilon',
yaxis_title='Accuracy',
title=f'Accuracy on Attack Data (Norm={norm})',
showlegend=True
)
fig.show()
from tabulate import tabulate
from functions.utils import evaluate_shift_consistency
model_consistency_dict = evaluate_shift_consistency(models,list(model_names.keys()), x_test)
simple_FC_2_256
100%|██████████| 100/100 [00:02<00:00, 44.97it/s]
simple_Conv_10_512
100%|██████████| 100/100 [00:36<00:00, 2.77it/s]
simple_Conv_NL
100%|██████████| 100/100 [00:07<00:00, 13.93it/s]
simple_Conv_max
100%|██████████| 100/100 [00:11<00:00, 8.53it/s]
simple_FC_3_512
100%|██████████| 100/100 [00:03<00:00, 32.42it/s]
simple_Conv_12_2048
100%|██████████| 100/100 [04:16<00:00, 2.56s/it]
simple__RNN
100%|██████████| 100/100 [00:10<00:00, 9.94it/s]
simple_FC_2
100%|██████████| 100/100 [00:02<00:00, 39.49it/s]
simple_Conv_2
100%|██████████| 100/100 [00:03<00:00, 29.72it/s]
table = []
for model_name, model_consistency in model_consistency_dict.items():
table.append([model_name, "{:.2f}%".format(100. * model_consistency)])
headers = ["Model Name", "Consistency"]
print(tabulate(table, headers, tablefmt="grid"))
+---------------------+---------------+ | Model Name | Consistency | +=====================+===============+ | simple_FC_2_256 | 36.45% | +---------------------+---------------+ | simple_Conv_10_512 | 54.91% | +---------------------+---------------+ | simple_Conv_NL | 58.05% | +---------------------+---------------+ | simple_Conv_max | 54.32% | +---------------------+---------------+ | simple_FC_3_512 | 37.90% | +---------------------+---------------+ | simple_Conv_12_2048 | 58.02% | +---------------------+---------------+ | simple__RNN | 43.86% | +---------------------+---------------+ | simple_FC_2 | 29.00% | +---------------------+---------------+ | simple_Conv_2 | 30.64% | +---------------------+---------------+
The difference isn't very large here (still for the models used in the paper the scores are smaller for the FCNsthan for the CNNs). If the consistency score should be inversely related to the invariance, then the order of robustness should be: simple_FC_2, simple_Conv_2, simple_FC_2_256,simple_FC_3_512, simple__RNN, simple_Conv_max,simple_Conv_10_512, simple_Conv_12_2048,simple_Conv_NL While the actual order of robustness for example for norm=∞ and epsilon=3: simple_Conv_2,simple_FC_2, simple_FC_2_256,simple_FC_3_512,simple_Conv_12_2048, simple__RNN, ,simple_Conv_10_512, ,simple_Conv_NL,simple_Conv_max
The models used in the paper, respect this order.
This part is also not a reproduction, just like we did with MNIST 2, it’s a verification. We created the models here using scripts/fashion_mnist_gpu_create_models.py and created the attack data for those models using scripts/fashion_mnist_gpu_create_attack_data.py. Where we used the FGM attack instead of the PGD norm = 2 (for the FGM we produced the attack on the full data, as it’s much faster to produce). We will also see the models, their accuracies and losses for the training. For the pgd we didn’t limit the iterations to 10 but to 100 and we left the other parameters to default
LRZ = False
if LRZ:
base_path = "/dss/dsshome1/lxc0C/apdl010/Analysis-of-new-phenomena-in-deep-learning/Reproduction"
else:
base_path = "models/fashion_mnist_2"
simple_FC = tf.keras.models.load_model(base_path + '/model_0.h5', compile=False)
simple_Conv = tf.keras.models.load_model(base_path+'/model_1.h5', compile=False )
simple_Conv_NL = tf.keras.models.load_model(base_path+'/model_2.h5', compile=False )
simple_Conv_max = tf.keras.models.load_model(base_path+'/model_3.h5', compile=False )
simple_FC_ = tf.keras.models.load_model(base_path+'/model_4.h5', compile=False )
simple_Conv_ = tf.keras.models.load_model(base_path+'/model_5.h5', compile=False )
simple__RNN = tf.keras.models.load_model(base_path+'/model_6.h5', compile=False )
simple_FC_2 = tf.keras.models.load_model(base_path+'/model_7.h5', compile=False )
simple_Conv_2 = tf.keras.models.load_model(base_path+'/model_8.h5', compile=False )
with open(base_path + '/acc_array.pkl', 'rb') as file:
acc_array = pickle.load(file)
with open(base_path + '/loss_array.pkl', 'rb') as file:
loss_array = pickle.load(file)
models = [simple_FC, simple_Conv, simple_Conv_NL, simple_Conv_max, simple_FC_,simple_Conv_, simple__RNN,simple_FC_2,simple_Conv_2]
models = [[models[i]] + [acc_array[i][0]] + [acc_array[i][1]] + [loss_array[i][0]] + [loss_array[i][1]] for i in range(len(models))]
visualkeras.layered_view(simple_FC, legend=True)
visualkeras.layered_view(simple_Conv, legend=True)
attack_params = [[np.inf, [0.05, 0.1, 0.15, 0.2, 0.25, 0.3]],[2, [0.5, 1, 1.5, 2.5, 3]]]
model_names = {'simple_FC_2_256':simple_FC, 'simple_Conv_10_512':simple_Conv, 'simple_Conv_NL':simple_Conv_NL,
'simple_Conv_max':simple_Conv_max, 'simple_FC_3_512':simple_FC, 'simple_Conv_12_2048':simple_Conv,'simple__RNN':simple__RNN , 'simple_FC_2':simple_FC_2, 'simple_Conv_2':simple_Conv_2}
import plotly.graph_objects as go
import plotly.io as pio
fig_traces = []
for model_name, (_, test_acc, train_acc,_,_) in zip(model_names.keys(), models):
trace_train = go.Scatter(x=np.arange(len(train_acc)), y=train_acc, mode='lines', name=f'{model_name} (Train)')
trace_test = go.Scatter(x=np.arange(len(test_acc)), y=test_acc, mode='lines', name=f'{model_name} (Test)')
fig_traces.extend([trace_train, trace_test])
layout = go.Layout(
xaxis=dict(title='Epoch'),
yaxis=dict(title='Accuracy'),
title='Training and Validation Accuracy',
showlegend=True
)
fig = go.Figure(data=fig_traces, layout=layout)
pio.show(fig)
fig_traces = []
for model_name, (_,_,_,test_loss,train_loss) in zip(model_names.keys(), models):
trace_train = go.Scatter(x=np.arange(len(train_loss)), y=train_loss, mode='lines', name=f'{model_name} (Train)')
trace_test = go.Scatter(x=np.arange(len(test_loss)), y=test_loss, mode='lines', name=f'{model_name} (Test)')
fig_traces.extend([trace_train, trace_test])
layout = go.Layout(
xaxis=dict(title='Epoch'),
yaxis=dict(title='Loss'),
title='Training and Validation Loss',
showlegend=True
)
fig = go.Figure(data=fig_traces, layout=layout)
pio.show(fig)
base_path = "data/adversarial_fashion_data"
# Load the attack data
attack_data = {}
attack_params = [[np.inf, [0.05, 0.1, 0.15, 0.2, 0.25, 0.3]],[2, [0.5, 1, 1.5, 2.5, 3]]]
model_names = {'simple_FC':simple_FC, 'simple_Conv':simple_Conv, 'simple_Conv_NL':simple_Conv_NL,
'simple_Conv_max':simple_Conv_max, 'simple_FC_':simple_FC, 'simple_Conv_':simple_Conv,
'simple__RNN':simple__RNN , 'simple_FC_2':simple_FC_2, 'simple_Conv_2':simple_Conv_2}
for model_name, model in zip(model_names,models):
for norm, epsilons in attack_params:
for epsilon in epsilons:
for data_type in ["train", "test"]:
if norm == np.inf:
attack_name = "ProjectedGradientDescentTensorFlowV2"
else:
attack_name = "FastGradientMethod"
file_name = base_path + f"/{model_name}_{attack_name}_{epsilon}_{data_type}.npz"
try:
attack_data[(model_name, norm, epsilon,data_type)] = (model,np.load(file_name))
except FileNotFoundError:
print(file_name + " data not found ")
norms = [attack_params[0][0], attack_params[1][0]]
epsilons = {
attack_params[0][0]: attack_params[0][1],
attack_params[1][0]: attack_params[1][1]
}
# Getting accuracies on normal data (epsilon = 0 as a notation)
accuracy_data = {norm: {model: [] for model in list(model_names.keys())} for norm in norms}
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(x_test,verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_test, axis=1)) / len(y_test)
accuracy_data[norm][model_name].append((0, accuracy))
else:
continue
# Getting accuracies on adversarial data (epsilon > 0)
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "test":
model = model[0]
predictions = model.predict(data[f'x_{data_type}_attack'],verbose=0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(data[f'y_{data_type}_attack'], axis=1)) / len(data[f'y_{data_type}_attack'])
accuracy_data[norm][model_name].append((epsilon, accuracy))
else:
continue
100%|██████████| 198/198 [12:44<00:00, 3.86s/it] 100%|██████████| 198/198 [05:01<00:00, 1.52s/it]
model_names = {'simple_FC': simple_FC, 'simple_Conv': simple_Conv, 'simple_Conv_NL': simple_Conv_NL,
'simple_Conv_max': simple_Conv_max, 'simple_FC_': simple_FC, 'simple_Conv_': simple_Conv,
'simple__RNN': simple__RNN, 'simple_FC_2': simple_FC_2, 'simple_Conv_2': simple_Conv_2}
for norm in norms:
plt.figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
if model_name == 'simple_FC':
model_name = 'simple_FC_2_256'
if model_name == 'simple_FC_':
model_name = 'simple_FC_3_512'
if model_name == 'simple_Conv':
model_name = 'simple_Conv_10_512'
if model_name == 'simple_Conv_':
model_name = 'simple_Conv_12_2048'
eps, accs = zip(*accuracies)
plt.plot(eps, accs, marker='o', label=model_name)
plt.xlabel('Epsilon')
plt.ylabel('Accuracy')
plt.title(f"Accuracy on Attack Data (Norm={norm})")
plt.legend()
plt.show()
import plotly.graph_objects as go
for norm in norms:
fig = go.Figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
if model_name == 'simple_FC':
model_name = 'simple_FC_2_256'
if model_name == 'simple_FC_':
model_name = 'simple_FC_3_512'
if model_name == 'simple_Conv':
model_name = 'simple_Conv_10_512'
if model_name == 'simple_Conv_':
model_name = 'simple_Conv_12_2048'
# Set the line style based on the model name
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=eps, y=accs, mode='lines+markers', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis_title='Epsilon',
yaxis_title='Accuracy',
title=f'Accuracy on Attack Data (Norm={norm})',
showlegend=True
)
fig.show()
The difference between the FCNs and the CNNs (that are used in the paper) is hard to distinguish compared to MNIST dataset. Which is expected as it's the case for the results of the paper. Except for epsilon bigger than one, for norm 2 (there is almost no difference). Some observations that can be explored during the extension phase: max pooling here has the worst robustness. Conv_2 is still robust even for this dataset. And RNN is the most robust.
accuracy_data = {norm: {model: [] for model in list(model_names.keys())} for norm in norms}
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "train":
model = model[0]
predictions = model.predict(x_train, verbose = 0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(y_train, axis=1)) / len(y_train)
accuracy_data[norm][model_name].append((0, accuracy))
else:
continue
for (model_name, norm, epsilon, data_type), (model, data) in tqdm(attack_data.items()):
if data_type == "train":
model = model[0]
predictions = model.predict(data[f'x_{data_type}_attack'], verbose = 0)
accuracy = np.sum(np.argmax(predictions, axis=1) == np.argmax(data[f'y_{data_type}_attack'], axis=1)) / len(data[f'y_{data_type}_attack'])
accuracy_data[norm][model_name].append((epsilon, accuracy))
else:
continue
100%|██████████| 198/198 [1:07:33<00:00, 20.47s/it] 100%|██████████| 198/198 [12:24<00:00, 3.76s/it]
for norm in norms:
plt.figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
if model_name == 'simple_FC':
model_name = 'simple_FC_2_256'
if model_name == 'simple_FC_':
model_name = 'simple_FC_3_512'
if model_name == 'simple_Conv':
model_name = 'simple_Conv_10_512'
if model_name == 'simple_Conv_':
model_name = 'simple_Conv_12_2048'
eps, accs = zip(*accuracies)
plt.plot(eps, accs, marker='o', label=model_name)
plt.xlabel('Epsilon')
plt.ylabel('Accuracy')
plt.title(f"Accuracy on Attack Data (Norm={norm})")
plt.legend()
plt.show()
for norm in norms:
fig = go.Figure()
for model_name in model_names:
accuracies = accuracy_data[norm][model_name]
eps, accs = zip(*accuracies)
if model_name == 'simple_FC':
model_name = 'simple_FC_2_256'
if model_name == 'simple_FC_':
model_name = 'simple_FC_3_512'
if model_name == 'simple_Conv':
model_name = 'simple_Conv_10_512'
if model_name == 'simple_Conv_':
model_name = 'simple_Conv_12_2048'
# Set the line style based on the model name
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=eps, y=accs, mode='lines+markers', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis_title='Epsilon',
yaxis_title='Accuracy',
title=f'Accuracy on Attack Data (Norm={norm})',
showlegend=True
)
fig.show()
shifts = range(10, -11, -1)
results = []
for model, model_name in zip(models, model_names):
model = model[0]
accuracy_axis0 = evaluate_shift_invariance(model, x_test, y_test, shifts, axis=0)
accuracy_axis1 = evaluate_shift_invariance(model, x_test, y_test, shifts, axis=1)
if model_name == 'simple_FC':
model_name = 'simple_FC_2_256'
if model_name == 'simple_FC_':
model_name = 'simple_FC_3_512'
if model_name == 'simple_Conv':
model_name = 'simple_Conv_10_512'
if model_name == 'simple_Conv_':
model_name = 'simple_Conv_12_2048'
results.append((model_name, accuracy_axis0, accuracy_axis1))
100%|██████████| 21/21 [00:16<00:00, 1.26it/s] 100%|██████████| 21/21 [00:14<00:00, 1.46it/s] 100%|██████████| 21/21 [01:53<00:00, 5.43s/it] 100%|██████████| 21/21 [01:49<00:00, 5.21s/it] 100%|██████████| 21/21 [01:45<00:00, 5.03s/it] 100%|██████████| 21/21 [01:47<00:00, 5.10s/it] 100%|██████████| 21/21 [01:44<00:00, 4.99s/it] 100%|██████████| 21/21 [01:44<00:00, 4.96s/it] 100%|██████████| 21/21 [00:18<00:00, 1.14it/s] 100%|██████████| 21/21 [00:18<00:00, 1.12it/s] 100%|██████████| 21/21 [09:13<00:00, 26.38s/it] 100%|██████████| 21/21 [09:29<00:00, 27.13s/it] 100%|██████████| 21/21 [01:38<00:00, 4.67s/it] 100%|██████████| 21/21 [01:43<00:00, 4.91s/it] 100%|██████████| 21/21 [00:14<00:00, 1.50it/s] 100%|██████████| 21/21 [00:21<00:00, 1.04s/it] 100%|██████████| 21/21 [00:32<00:00, 1.55s/it] 100%|██████████| 21/21 [00:35<00:00, 1.70s/it]
#fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(8, 10))
for model_name, accuracy_axis0, accuracy_axis1 in results:
#axes[0].plot(shifts, accuracy_axis0, label=model_name)
plt.plot(shifts, accuracy_axis1, label=model_name)
plt.xlabel('Shift')
plt.ylabel('Accuracy')
plt.legend()
plt.title('Shift Invariance Evaluation')
#fig = plt.figure(figsize=(20, 20))
#plt.tight_layout()
plt.show()
fig = go.Figure()
for model_name, accuracy_axis0, accuracy_axis1 in results:
shifts_abs = []
results_abs_k = []
for k in range(len(shifts) // 2 + 1):
shifts_abs.append(shifts[k])
results_abs_k.append((accuracy_axis1[k] + accuracy_axis1[-k - 1]) / 2)
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=shifts_abs, y=results_abs_k, mode='lines', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis=dict(title='Shift'),
yaxis=dict(title='Accuracy'),
title='Shift Invariance Evaluation',
showlegend=True
)
fig.show()
This part is to compare Shift invariance and padding size . We used scripts/padding_plot.py to train the models. We trained different models using different padding sizes. To check the training accuracy and loss, you can use tensorboard and .logs folder. Here are the models that were trained:
import os
base_path = "data/padding/padding_experiment"
accuracy_data_path = os.path.join(base_path, "accuracy_data.pkl")
with open(accuracy_data_path, "rb") as f:
accuracy_data = pickle.load(f)
def circular_padding(x, padding_size):
return tf.pad(x, [[0, 0], [padding_size, padding_size], [padding_size, padding_size], [0, 0]], mode='SYMMETRIC')
models_ = []
model_names_ = []
model_dir = base_path
for filename in os.listdir(model_dir):
if filename.endswith(".h5"):
model_path = os.path.join(model_dir, filename)
model = tf.keras.models.load_model(model_path, compile=False)
models_.append(model)
model_names_.append(filename[:-3]) # Remove the ".h5" extension from the filename
print("The trained models:")
print(model_names_)
The trained models: ['simple_Conv_0_2048', 'simple_Conv_0_512', 'simple_Conv_10_2048', 'simple_Conv_10_512', 'simple_Conv_14_2048', 'simple_Conv_14_512', 'simple_Conv_2_2048', 'simple_Conv_2_512', 'simple_Conv_4_2048', 'simple_Conv_4_512', 'simple_Conv_6_2048', 'simple_Conv_6_512', 'simple_Conv_8_2048', 'simple_Conv_8_512', 'simple_Conv_NL', 'simple_FC_2_256', 'simple_FC_3_256']
# Plot the accuracy related to shift
import matplotlib.pyplot as plt
shifts = list(range(10, -11, -1))
plt.figure(figsize=(12, 10))
for model_name, accuracy_axis in accuracy_data.items():
plt.plot(shifts, accuracy_axis, label=f"Model: {model_name}")
plt.title(f"Shift Invariance")
plt.xlabel("Shift")
plt.ylabel("Accuracy")
plt.legend()
plt.grid(True)
plt.show()
import plotly.graph_objects as go
shifts = list(range(10, -11, -1))
fig = go.Figure()
for model_name, accuracy_axis in accuracy_data.items():
fig.add_trace(go.Scatter(x=shifts, y=accuracy_axis, mode='lines', name=f"Model: {model_name}"))
fig.update_layout(
title=f"Shift Invariance ",
xaxis_title="Shift",
yaxis_title="Accuracy",
legend=dict(x=0, y=1),
xaxis=dict(showgrid=True, gridwidth=1, gridcolor='lightgray'),
yaxis=dict(showgrid=True, gridwidth=1, gridcolor='lightgray'),
width=800,
height=600
)
fig.show()
fig = go.Figure()
for model_name, accuracy_axis in accuracy_data.items():
shifts_abs = []
results_abs_k = []
for k in range(len(shifts) // 2 + 1):
shifts_abs.append(shifts[k])
results_abs_k.append((accuracy_axis[k] + accuracy_axis[-k - 1]) / 2)
line_dash = 'solid' if 'FC' not in model_name else 'dash'
fig.add_trace(go.Scatter(x=shifts_abs, y=results_abs_k, mode='lines', name=model_name, line=dict(dash=line_dash)))
fig.update_layout(
xaxis=dict(title='Shift'),
yaxis=dict(title='Accuracy'),
title='Shift Invariance Evaluation',
showlegend=True
)
fig.show()
from functions.utils import extract_padding
accuracy__values = []
padding_values = []
accuracy__values_ = []
padding_values_ = []
for model_name, accuracy_axis in accuracy_data.items():
shift = 7
padding = extract_padding(model_name)
if padding == 0:
continue
layer_size = int(model_name.split('_')[3])
if layer_size == 512:
padding_values.append(padding)
accuracy__values.append((accuracy_axis[shift] + accuracy_axis[-shift - 1]) / 2)
if layer_size == 2048:
padding_values_.append(padding)
accuracy__values_.append((accuracy_axis[shift] + accuracy_axis[-shift - 1]) / 2)
# Plot the data
plt.plot(padding_values, accuracy__values, 'o-', label="layer size: 512" )
plt.plot(padding_values_, accuracy__values_, '*-', label="layer size: 2048" )
plt.xlabel('Padding')
plt.ylabel(f'Shift -{shift} Accuracy')
plt.title('Model Shift Accuracy vs. Padding')
plt.legend()
plt.grid(True)
plt.show()
We can't really conclude that higher padding makes the model more shift invariant.
Now let's use the paper's consistency score method. First let's plot examples of the shifts
from functions.utils import plot_shift_examples
plot_shift_examples(x_test,n=3)
models_ = [[_] for _ in models_]
model_consistency_dict = evaluate_shift_consistency(models_,model_names_, x_test,10)
simple_Conv_0_2048
100%|██████████| 100/100 [00:07<00:00, 13.95it/s]
simple_Conv_0_512
100%|██████████| 100/100 [00:04<00:00, 23.78it/s]
simple_Conv_10_2048
100%|██████████| 100/100 [00:13<00:00, 7.57it/s]
simple_Conv_10_512
100%|██████████| 100/100 [00:06<00:00, 16.38it/s]
simple_Conv_14_2048
100%|██████████| 100/100 [00:17<00:00, 5.71it/s]
simple_Conv_14_512
100%|██████████| 100/100 [00:07<00:00, 13.70it/s]
simple_Conv_2_2048
100%|██████████| 100/100 [00:06<00:00, 15.64it/s]
simple_Conv_2_512
100%|██████████| 100/100 [00:04<00:00, 21.58it/s]
simple_Conv_4_2048
100%|██████████| 100/100 [00:08<00:00, 11.52it/s]
simple_Conv_4_512
100%|██████████| 100/100 [00:05<00:00, 16.77it/s]
simple_Conv_6_2048
100%|██████████| 100/100 [00:09<00:00, 10.49it/s]
simple_Conv_6_512
100%|██████████| 100/100 [00:04<00:00, 20.37it/s]
simple_Conv_8_2048
100%|██████████| 100/100 [00:11<00:00, 8.44it/s]
simple_Conv_8_512
100%|██████████| 100/100 [00:05<00:00, 18.22it/s]
simple_Conv_NL
100%|██████████| 100/100 [00:05<00:00, 19.79it/s]
simple_FC_2_256
100%|██████████| 100/100 [00:03<00:00, 27.33it/s]
simple_FC_3_256
100%|██████████| 100/100 [00:04<00:00, 23.73it/s]
table = []
for model_name, model_consistency in model_consistency_dict.items():
padding = extract_padding(model_name)
table.append([model_name, padding, "{:.2f}%".format(100. * model_consistency)])
# Table headers
headers = ["Model Name", "Padding", "Consistency"]
# Print the table
print(tabulate(table, headers, tablefmt="grid"))
+---------------------+-----------+---------------+ | Model Name | Padding | Consistency | +=====================+===========+===============+ | simple_Conv_0_2048 | 0 | 59.77% | +---------------------+-----------+---------------+ | simple_Conv_0_512 | 0 | 56.89% | +---------------------+-----------+---------------+ | simple_Conv_10_2048 | 10 | 50.25% | +---------------------+-----------+---------------+ | simple_Conv_10_512 | 10 | 51.04% | +---------------------+-----------+---------------+ | simple_Conv_14_2048 | 14 | 45.54% | +---------------------+-----------+---------------+ | simple_Conv_14_512 | 14 | 45.76% | +---------------------+-----------+---------------+ | simple_Conv_2_2048 | 2 | 42.44% | +---------------------+-----------+---------------+ | simple_Conv_2_512 | 2 | 43.79% | +---------------------+-----------+---------------+ | simple_Conv_4_2048 | 4 | 49.09% | +---------------------+-----------+---------------+ | simple_Conv_4_512 | 4 | 49.05% | +---------------------+-----------+---------------+ | simple_Conv_6_2048 | 6 | 56.31% | +---------------------+-----------+---------------+ | simple_Conv_6_512 | 6 | 53.89% | +---------------------+-----------+---------------+ | simple_Conv_8_2048 | 8 | 45.63% | +---------------------+-----------+---------------+ | simple_Conv_8_512 | 8 | 44.83% | +---------------------+-----------+---------------+ | simple_Conv_NL | 0 | 69.28% | +---------------------+-----------+---------------+ | simple_FC_2_256 | 0 | 27.23% | +---------------------+-----------+---------------+ | simple_FC_3_256 | 0 | 31.72% | +---------------------+-----------+---------------+
# Create dicts for padding and consistency values
padding_values = {"512":[],"2048":[]}
consistency_values = {"512":[],"2048":[]}
# Populate the lists
for model_name, model_consistency in model_consistency_dict.items():
padding = extract_padding(model_name)
if padding == 0:
continue
layer_size = model_name.split('_')[3]
padding_values[layer_size].append(padding)
consistency_values[layer_size].append(model_consistency)
# Plot the data
plt.plot(padding_values["512"], consistency_values["512"], 'o', label="layer size: 512" )
plt.plot(padding_values["2048"], consistency_values["2048"], 'o', label="layer size: 2048" )
plt.xlabel('Padding')
plt.ylabel('Model Consistency')
plt.title('Model Consistency vs. Padding')
plt.legend()
plt.grid(True)
plt.show()
Same conclusion as with the shifting method.
fig, axes = plt.subplots(1, 2, figsize=(12, 6))
axes[0].plot(padding_values_, accuracy__values, 'o-', label="layer size: 512" )
axes[0].plot(padding_values_, accuracy__values_, '*-', label="layer size: 2048" )
axes[0].set_xlabel('Padding')
axes[0].set_ylabel('Model Accuracy')
axes[0].set_title('Model Shift Accuracy vs. Padding')
axes[0].legend()
axes[0].grid(True)
axes[1].plot(padding_values["512"], consistency_values["512"], 'o', label="layer size: 512" )
axes[1].plot(padding_values["2048"], consistency_values["2048"], 'o', label="layer size: 2048" )
axes[1].set_xlabel('Padding')
axes[1].set_ylabel('Model Consistency')
axes[1].set_title('Model Consistency vs. Padding')
axes[1].legend()
axes[1].grid(True)
plt.tight_layout()
plt.show()
While trying to understand and reproduce the results of the paper, we added multiple verifications and little tweaks (mini-extensions). The goal is to continue exploring the outliers or unexpected results. RNN structures or transformers. The transferability of the attacks and maybe some defense to the attacks.
Singla, Vasu, et al. Shift Invariance Can Reduce Adversarial Robustness. ArXiv.org,22 Nov 2021 https://arxiv.org/pdf/2103.02695.pdf.
Alireza, et al. “Revisiting DeepFool: Generalization and Improvement.” ArXiv.org, 22 Mar. 2023, arxiv.org/abs/2303.12481, https://doi.org/10.48550/arXiv.2303.12481.
Rony, Jérôme, et al. “Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses.” ArXiv.org, 3 Apr. 2019, arxiv.org/abs/1811.09600.
Xiao, Han, et al. “Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms.” ArXiv:1708.07747 [Cs, Stat], 15 Sept. 2017, arxiv.org/abs/1708.07747.
Zhang, Richard. “Making Convolutional Networks Shift-Invariant Again.” ArXiv.org, 8 June 2019, arxiv.org/abs/1904.11486.
“Welcome to the Adversarial Robustness Toolbox — Adversarial Robustness Toolbox 1.7.0 Documentation.” Adversarial-Robustness-Toolbox.readthedocs.io, adversarial-robustness-toolbox.readthedocs.io/en/latest/.
“Trusted-AI/Adversarial-Robustness-Toolbox.” GitHub, 1 July 2021, github.com/Trusted-AI/adversarial-robustness-toolbox.